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1.  Introduction 


An  AutoreKresslve  process  {y^,  t  ■  0,  >  1,...)  of  order  p  la  defined  by 

(1.1)  ^  i  "  ^  t  ^ . 

1-0  ^ 

tfhere  Yq  **  1  and  la  a  aequence  of  uncorrelated  randoa  varlablea  with  mean 

2  Pi 

0  and  coesBon  variance  a  .  We  assume  that  the  roots  of  I  Z  -  0  are  outside 

1-0  ^ 

the  unit  disc.  The  process  (I.l)  la  completely  specified  by 
2 

$  z  (y  , . . . ,  y  ,  a  )*  when  the  €  are  assumed  to  be  normally  distributed. 

A  P  t 

Throughout  thin  paper  we  shall  assume  normality  of 

Usually  statistical  Inference  la  based  on  a  set  of  T  consecutive  observations 
on  y^.  Let 

(1.2)  y  —  (y^*  y2****»  y*j'^  * 

and  let  P  be  a  permutation  matrix  such  that  Py.  -  (s',  m*)*,  tdtere  a  la  a 

(T-m)  K  1  vector  and  m  Is  an  mnl  vector,  with  the  ordering  In  a  and  n  preserved. 

Suppose  only  observations  In  s  are  available  and  those  In  m  are  missing.  Our 

goal  here  Is  to  obtain  m.axlanin  likelihood  estimates  of  f. 

For  any  T*!  n.atrlx  C,  let  us  define  C  ,  C  .  C  and  C _  to  be  the 

^  »*-oo  -^sm  -ms  -rm 

(T-m)  ■  (T-n),  (T-n)  *  a,  m  «  (T-m)  and  m^m  matrices,  respectively,  satisfying 


(1.3) 


/ 


P  C  P'  - 


C 

— as 


C 

ivms 


For  the  rest  of  this  paper,  let  f(y|#)  denote  the  probability  density  function 
of  y,  f(jB|f)  denote  the  probability  density  function  of  j|,  f(£lj>«  #)  denote  the 
conditional  probability  density  function  of  m  given  a,  log  f(y|d)  denote  the  log 


2 


likelihood  function  based  on  ^  and  log  f(aU)  denote  the  log  likelihood  function 
beeed  on  e.  We  aaeune  that  the  iMxlnua  likelihood  eolutlone  satisfy 


(l.A) 


3  log  f(sk) 


2.  Sosa  basic  results 


gssune  that  y  la  distributed  as  sultlvarlate  normal  with  mean  JQ  and 
covariance  matrix  that  ls«  f(y|4)  Is  given  by 


(2.1) 


f(v|e)  -  —  - - e*P 

111 


Then  -  (s’,  a’)’  is  distributed  as  multivariate  normal  with  mean  0  and 

•s>w 

covariance  matrix  HI'-  Since  P  P*  -  **  ‘^entity  matrix. 


(2.2) 


(P  r  P’)'^  -  P  P’ 
^  ^  ^ 


-  P  M  P* 

(,2 - 


•here  -  M.  -nd^.,  and  are  as  defined  by  (1.3).  Also,  by 


(1.3),  we  get 


‘  4  i  ■’(  >  ^ 
^  ™  »|  s 
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Tb«r«forn  froa  (2.2)  and  (2.3),  It  follova  th.tt 


txpraaslona  (2.5)  and  (2.6)  will  ba  used  In  the  following  sections.  Though  (2.8) 


gives  the  expression  for  the  probability  density  function  of  s,  we  will  not  use 
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It  to  obtain  th«  .core  function  -  .  «*“•  “»•  •‘^Uclty  of 

a  log  f(2|4) 

- —  and  Lcnaa  I  In  section  3.  We  will  uae  (2.8)  In  proving  the  aayaptotlc 

properties  of  the  estlnates  in  a  subsequent  paper.  Under  suitable  conditions, 
the  estlnates  of  4  based  on  the  Mewton-Raphson  nethod  and  the  method  of  scoring 
srs  shown  to  be  /T-m  -  consistent,  asynptotlcally  normal  and  one-step  ssymptotlcally 
efficient  If  the  Initial  estlnates  are  i^-n  -  consistent. 


3.  Estlnatlon 


(3.1) 


■  (Yjt  •  •  • .  Yp)' 


(3.2) 


Y..  -  d.Y’)*. 


Than  (see  Anderson  (1971),  sec  6.2,  and  Box  and  Jenkins  (1976),  sec  7. A. 3) 


(3.3)  log  f(yl4)  ■  -  7  log  (2it  o^)  +  i  log  Im] - ^  y'  M  y 


-  -  I  log  (2x  o^)  +  -j  log  1m| - -  y’  D  y  , 

2  Z  -  2^Z  ... 


where  the  elements  m  of  the  T*?  natrlx  M  is  given  by 

AC  0^ 


(3.4) 


"st  "  ^+l-t,  T-fl-i 
nln(s.t)-l 


nln(s.t)-l 

"  Jo  "jVl-tl. 

H*-t| 

"  Jo  ''i  Vl.-t|. 


s,  t  •  1,...,  p, 

max  (s,t)  p4-l, 

nln  (s,t)  <  T-p, 

|s-t|  -  0,1 . . 

|s-t|  -  p+1,.... 
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:v  - 


6 


Proof.  The  refmlt  follown  Irnmedletely  from 

S  lop,  f(nl«,^) 

'  - 3r^  I  5.?1  -  »• 


Cl.  101 


It  te  deer  from  (3.7)  -  (3.9)  thet 


(3.11) 


3  loR  f(«U)  3  loR  |m|  .  p  _ 

-•  V'~  ■  ^  ■  7  iV* ' 


j  -  1,. . . ,  p. 


(3.12) 


3  log  f(8|*) 


—  -  -  y'  E  [d  is.  ♦  1  Y 

20^  20^  L-  ~  -  J  -U 


Ths  term  log  Im|  Is  0(1)  (see  Hannan  (1973),  e.g.)  %fhile  d.,  la  0  (T).  The 
-  Ij  P 

effect  of  neglecting  log  !m  |  Is  negligible  for  moderate  or  large  T,  and  ve  shall 
neglect  log  |m|  and  other  negligible  terms  henceforth.  From  (3.11)  and  (3.12), 
it  follows  that  the  likelihood  equations  are  given  by 


(3.13) 


(3.14)  -  I  Y*  fFd  I  . 

T  I  ~  1  ».ll 

When  there  are  no  missing  observations,  sTd  |  e.fl*  d  ...  does  not  involve 

I.  (t+ltj"*!  ^  **1  It+ltj'*'! 

unknown  parameters.  Then  the  equations  are  linear  In  Y^t  1  ■  1,...,  p,  and  are 
the  Yulr-V?alker  equations.  When  missing  observations  do  occur, 
sj^d^j  (  8,  ♦j  Involves  unknown  parameters  and  (3.13)  and  (3.14)  are 
highly  non-llncar  In  the  unknown  parameters.  In  fact,  frcn  (2.5)  and  (2.6), 

(3.15)  ^[‘‘g+l.J-H  I  "’♦] 

“  £*  [^-g+l.J+l^ss  "  ^-g+l.J+l^sm  5 
-  K’  (A^,  _-)  4  V  (A  _  iH  • 


,J41^88  "  ^-g+l.J+l^sm  5 


rr  (A  .  .)  M 


7 


wh«r«  K  •  M  M  ,  and  the  natrices  involved  are  as  defined  In  (1.3),  (3.4) 
end  (3.6).  Therefore  solutions  of  (3.13)  and  (3.14)  are  not  straightforvard 
and  iterative  procedures  have  to  be  used. 

Ue  propose  the  following  three  methods  of  solving  (3.13)  and  (3.14): 
the  EM  algorithm,  the  Newton-Raphson  method  and  the  method  of  scoring. 

a.  The  Pt  algorithm.  Since  ^  la  to  be  estimated.  It  is  natural  that 

one  replace  eT*  |  s,  4"]  In  (3.13)  and  (3.14)  by  eF*  I  s,  ♦,!,  where  4,  is 

—  -1'*  «.i 

some  estimated  value  of  4,  and  obtain  4...  Iteratively  by  solving 

<ae  « 

(3.14)  ^  ^"^g^l+l  '•[‘’g+l,j+l  ^ 


J  ■  1,...,  p. 


(3.17) 


Vi^  "T  ‘'[2  I  2'  !i]<!p\ 


Here  (y  )  and  (y  )  denote  the  eatim.3te8  of  y  and  y  ,  respectively,  at 
R  j  •»u  j  g 

the  J-th  iteration.  A.a  shown  in  Tan  (1479),  the  above  method  gives  the  same 
solutions  as  the  EM  algorithm  proposed  by  Dempster,  Laird  and  Rubin  (1977). 

b.  The  Newton-Raphson  Method.  From  (3.11)  and  (3.12),  we  obtain 


(3. IP)  0, 


3^  log  f(8l4)  1  P  ^  r 

■  7  'Jo  K  '  ;•  il 


J,  k*  1,...,  p. 


8 
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and 


(1.25) 


P^ltP+1 


=  E[0, 


p+l,p+l 


] 


o(i-J) 


'  2o^  l.Lo  ^N-H.j+l^om  ”nim  ’ 


where  o(k)  "  y^]  •  Thus,  the  method  of  Bcoring  leads  to  the  followlnR 

act  of  equations: 


0.26) 


1+1 


!i>  ■ 


3  log  fCaU) 


3^ 


3  log  ffal*) 

where  the  eleroenta  t  of  9  are  given  hv  (3.23)  -  (3.25),  and - — —  —  ■ 

I  j  ^  aA 

MS 

la  given  by  (3.11)  (without  the  first  term  on  the  right-hand  side)  and  (3.12). 

Ue  have  used  the  fact  that  J+l^  "  which  can  be 

approximated  by  T  nd-J)  for  moderate  or  large  T. 


4.  Comparison  of  the  methods  of  estimation 

The  estimates  of  >  based  on  the  Newton-Raphson  method,  the  method  of 
scoring  and  the  EM  algorithm  can  be  expressed  In  the  following  form 

3  log  f(aU) 


3  log  f(8U) 

where - ^ is  given  by  (3.11)  (without  the  first  term  on  the  right-hand 

side)  and  (3.12).  In  the  Newton-Raphson  method,  we  have 


I'.. 


(a 

.1+1 


!i> 


H  -  0, 

where  0  is  given  by  (3.1R)  -  (3.21).  In  the  method  of  scoring,  we  have 
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I  (A)  1b  ref<»rr«»<l  ro  hh  rh»»  lost  InforraBtlon  m.itrlx  In  OrrhBrd  and  Woodhtirv 
^•m  - 

(l‘'72).  It  follow*}  that 


E(A)  -  0, 


and  alRo 


lln 

T-rt**” 


F.(n) 


T-n 


1  Im 
T-ta~ 


f:(c) 

T-m 


lln 

T-nr“> 


I  («) 

T-n 


since  11m 


E(u)) 

f-n” 


11m  -  E 

T-nr<-»> 


^  3  log  f(y|d) 

T— m  3(^  3^  * 

I  («> 


(see  Box  and  Jenkins  (1976) 


section  7. A. 5).  In  general,  lln 


>»in  •• 
T-n 


is  nor  negligible.  For  example. 


when  p  "  1  in  (1.1)  and  the  prccesn  periodically  observed  for  n  time 

points  and  then  not  observed  for  two  time  points.  It  can  be  shown  that  (see 
Tan  (1979)) 


11m 

T-st*« 


I  (♦) 

■»n  - 
'  T-n 


ao 


?  ?  2  4 

o  (1  .2y^^)  (1  .  -  r^) 


-  Yj(l  +  7y 


ll\ 


-  Yj  (i  +  2yp 


where  \  -  (1  y*  +  Y^-  It  Is  easy  to  see  that  the  above  matrix  is  positive 
definite. 
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